Pharo process scheduling

On 30 Jul 2019, at 20:14, Alistair Grant <[hidden email]> wrote:

Hi Henrik,

I have to admit that I'm having a bit of trouble following what you're 
saying, so I'll make a few comments that I hope are helpful...

Just to start off, my understanding of the term "pre-emptive, 
cooperative" processing is:

- a higher priority process will interrupt a lower priority process and 
 run until it voluntarily waits (or is suspended or terminated).
- "cooperative" should be read as "unless the processes cooperate, one 
 bad actor can stop anyone else from getting the CPU".


On Tue, Jul 30, 2019 at 05:32:13AM -0500, Henrik Sperre Johansen wrote:

Alistair Grant wrote

The following produces what I was expecting:

| p x a index first |

Smalltalk vm processPreemptionYields: false.


Thanks again!
Alistair

Well... what your example shows, is that with processes
p50 (pri50) periodic, becoming runnable in time slot 2)
p40.1 (taking 3 time slots to complete)
p40.2 (taking 3 time slots to complete)

When preemption yields, you get processor time allocations like this:
[p40.1][p50][p40.2][p40.1][p40.2][p40.1][p40.2]

I think this might be demonstrating an incorrect expectation on your 
part.

Your use of the term "time slot" seems to imply that, within a given 
process priority, each process can expect roughly equal CPU time.  
However that isn't what is implied.  With processPreemptionYields: true 
*any* higher priority process becoming runnable will cause the currently 
executing process to be moved to the back of the queue.  In practice, 
where processes are running at a priority of less than 60, this means 
that with every mouse move, keyboard input, timer event, etc., the 
active process will be moved to the back of the queue.  The amount of 
time allocated to each process is effectively random.

So in the above example, assuming that the p50 process only becomes 
runnable once, and no other high priority process becomes runnable, we 
can expect:

[p40.1][p50][p40.2][p40.1]

With preemption NOT yielding, you get processor time allocations like this:
[p40.1][p50][p40.1][p40.1][p40.2][p40.2][p40.2]

I'd expect:

[p40.1][p50][p40.1][p40.2]

(running the example code used earlier in VW also supports this)

Which clearly contradicts the statements in the linked post
"Nothing changes within priorities.

I haven't looked up the original post, but it seems out of 
context.

What the setting controls is what
happens when a process is preempted by a higher-priority process.  The old
code did an implicit yield of the preempted process, destroying the
guarantee of cooperative scheduling within a priority."

This matches my understanding (remember that "cooperative" means that a 
process has to voluntarily relinquish the CPU).

Which (to me) would imply slot allocations when not yielding would be like
this:
[p40.1][p50][p40.1][p40.2][p40.1][p40.2][p40.2]

Maybe it's what is intended and the statements are inaccurate, or Pharo is
lacking some related changes, but it doesn't make for a good experience when
you have multiple processes requiring "real-time" execution running at the
same priority if you have to add explicit Processor yield's everywhere.  

Actually, this matches exactly my understanding of real-time 
processing.  E.g. in VAX/VMS there were two tiers of process priority:

- the low priority tier, e.g. 1 - 40 (I can't remember the actual 
 numbers) were time sliced.  So if multiple processes were at the same 
 priority, they could expect roughly equal amounts of CPU time.
- the high priority tier, e.g. 41 - 80, were cooperative.  So if 
 multiple processes were at the same priority, the second one had to 
 wait for the first to voluntarily give up the CPU.
 The expectation in this case was that any time a real-time process had 
 work to do, the time required would be reasonably short, and that it 
 was important to let it have the CPU until it had completed that work.



On Tue, Jul 30, 2019 at 05:53:52AM -0500, Henrik Sperre Johansen wrote:

Henrik Sperre Johansen wrote

Which clearly contradicts the statements in the linked post
"Nothing changes within priorities.  What the setting controls is what
happens when a process is preempted by a higher-priority process.  The old
code did an implicit yield of the preempted process, destroying the
guarantee of cooperative scheduling within a priority."

Or it could mean the understanding was that processes at the same priority
would have to explicitly yield for other processes of the same priority to
run also with preemptionyields = true.
... which, seems to be true.

It's just that we never see that in practice, even running at pri 80, as the
DelaySemaphoreScheduler (which runs at 80) is at the head of the list, and
gets woken every now and then.
Reducing it's pri to 70, and then doing the same test with threads at pri80
(but containing plenty of suspension points), we indeed get 
1 1 2 2 3 3 4 4 5 5 6 6 7 7 8 8 9 9 10 10

You get this result even if you don't lower the priority of the timer 
process.  If the sample code is run at priority 80 it will continue 
until it has finished, and the DelaySemaphoreScheduler has to wait for 
it to finish (which has other bad side effects, but we don't care about 
those here).

TIL, I guess :)

Cheers, 
Henry

Hope this helps,
Alistair