The Inbox: Kernel-jar.1368.mcz
Locked
 
|
A new version of Kernel was added to project The Inbox:
http://source.squeak.org/inbox/Kernel-jar.1368.mcz ==================== Summary ==================== Name: Kernel-jar.1368 Author: jar Time: 8 February 2021, 10:03:00.688403 pm UUID: 095338a7-d694-3e4c-97dd-1f33e4d08156 Ancestors: Kernel-eem.1367 Fix incorrect #priority: behavior blocking higher priority processes =============== Diff against Kernel-eem.1367 =============== Item was changed: ----- Method: Process>>priority: (in category 'accessing') ----- priority: anInteger "Set the receiver's priority to anInteger." (anInteger >= Processor lowestPriority and:[anInteger <= Processor highestPriority]) + ifTrue: [ + priority := anInteger. + self isActiveProcess ifTrue: [ [ ] forkAt: Processor highestPriority] ] - ifTrue: [priority := anInteger] ifFalse: [self error: 'Invalid priority: ', anInteger printString]! |
Re: The Inbox: Kernel-jar.1368.mcz
|
I propose the following change for consideration / discussion: modify
Process>>#priority behavior so that it guarantees no higher priority processes remain waiting before active process continues at a lower priority. The proposed fix only affects active process priority changes. Replacing the method by a primitive may also be considered. Impact: #valueUnpreemptively and #valueAt: methods are currently affected by the #priority:'s incorrect behavior and need to be amended in any case. This fix would cover both of them. If this fix is accepted the two methods should remove `Processor yield` line which is not working anyway. Related issue with examples: http://forum.world.st/Process-gt-gt-priority-behavior-grossly-incorrect-Transcript-losing-output-td5126836.html Test: KernelTests-jar.393 (The Inbox) ``` testPriority "test whether #priority: preempts active process to allow higher priority processes run" | val oldPriority | val := nil. oldPriority := Processor activePriority. Processor activeProcess priority: oldPriority + 2. [ val := false ] forkAt: oldPriority + 1. [ val := true ] forkAt: oldPriority. Processor activeProcess priority: oldPriority. self assert: val equals: Smalltalk vm processPreemptionYields ``` The test respects the current setting of #processPreemptionYields. -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
Re: The Inbox: Kernel-jar.1368.mcz
|
|
This post was updated on .
For some reason the proposed fix also fixes the Transcript losing output problem mentioned in
http://forum.world.st/Process-gt-gt-priority-behavior-grossly-incorrect-Transcript-losing-output-td5126836.html => it looks like the current #priority bug influences Transcript behavior and who knows what else... -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
Re: The Inbox: Kernel-jar.1368.mcz
|
|
Hi Jaromir,
> On Feb 8, 2021, at 2:32 PM, Jaromir <[hidden email]> wrote: > > For some reason the proposed fix also fixes the Transcript losing output > intermittently problem mentioned in > http://forum.world.st/Process-gt-gt-priority-behavior-grossly-incorrect-Transcript-losing-output-td5126836.html > => it looks like the current #priority bug influences also Transcript > behavior and who knows what else... That’s a good catch. What if instead of the fork you simply use yield? yield is now implemented as a primitive so it is far faster than creating a new process and resuming it. I’ll take a look at the yield primitive today and make sure it handles this use case. It’s great to have your insight and energy looking at this. Thank you!! Eliot _,,,^..^,,,_ (phone) |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
> What if instead of the fork you simply use yield? yield is now implemented
as a primitive so it is far faster than creating a new process and resuming it. Hi, thanks for your encouragement! I did try to use #yield but the primitive seems to check only the active priority queue (and assumes it can't even happen that higher priority processes are waiting while a lower priority process is active - which is probably a reasonable assumption - if it weren't for the #priority: behavior). So I guess either #priority: or #yield should be modified. Thanks, Jaromir -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
Re: The Inbox: Kernel-jar.1368.mcz
|
|
> On 2021-02-10, at 10:30 AM, Jaromir <[hidden email]> wrote: > I did try to use #yield but the primitive > seems to check only the active priority queue (and assumes it can't even > happen that higher priority processes are waiting while a lower priority > process is active - which is probably a reasonable assumption - if it > weren't for the #priority: behavior). So I guess either #priority: or #yield > should be modified. That doesn't seem right; the primitiveYield carefully transfers control to the result of `wakeHighestPriority()` which is certainly *meant* to find the highest priority process that is runnable. It's what I wrote the original version to do nearly 20 years ago.. tim -- tim Rowledge; [hidden email]; http://www.rowledge.org/tim Hackers have kernel knowledge. |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
On Wed, 10 Feb 2021, tim Rowledge wrote:
> > >> On 2021-02-10, at 10:30 AM, Jaromir <[hidden email]> wrote: >> I did try to use #yield but the primitive >> seems to check only the active priority queue (and assumes it can't even >> happen that higher priority processes are waiting while a lower priority >> process is active - which is probably a reasonable assumption - if it >> weren't for the #priority: behavior). So I guess either #priority: or #yield >> should be modified. > > That doesn't seem right; the primitiveYield carefully transfers control to the result of `wakeHighestPriority()` which is certainly *meant* to find the highest priority process that is runnable. It's what I wrote the original version to do nearly 20 years ago.. The problem is that when you change the priority via #priority:, the process will not be placed to the list corresponding to the new priority. Also, #yield is not optimal, because it will #yield instead of just keep running the current process with the new priority. IMO, the right solution is to create a new primitive if we want to support on-the-fly priority changes. Levente > > > tim > -- > tim Rowledge; [hidden email]; http://www.rowledge.org/tim > Hackers have kernel knowledge. |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
> the right solution is to create a new primitive if we want to support
on-the-fly priority changes. The system already does on-the-fly priority changes - implementing #valueUnpreemptively and #valueAt: - but they're actually not working properly due to this #priority: issue - they're trying to use #yield after changing the priority down but that really doesn't do anything. Jaromir -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
Re: The Inbox: Kernel-jar.1368.mcz
|
|
On Wed, Feb 10, 2021 at 11:27 AM Jaromir <[hidden email]> wrote: > the right solution is to create a new primitive if we want to support So it seems to me that updating the yield primitive to "do the right thing" is a good way to go. Agreed? If so, I should be able to get to that very soon. _,,,^..^,,,_ best, Eliot |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
On Wed, 10 Feb 2021, Eliot Miranda wrote:
> > > On Wed, Feb 10, 2021 at 11:27 AM Jaromir <[hidden email]> wrote: > > the right solution is to create a new primitive if we want to support > on-the-fly priority changes. > > The system already does on-the-fly priority changes - implementing > #valueUnpreemptively and #valueAt: - but they're actually not working > properly due to this #priority: issue - they're trying to use #yield after > changing the priority down but that really doesn't do anything. > > > So it seems to me that updating the yield primitive to "do the right thing" is a good way to go. Agreed? If so, I should be able to get to that very soon. increase it to 40 and then do something at that priority. #yield will stop the process and put it at the end of the list of processes with 40 priority (I think the current implementation doesn't check for priority changes and puts the process on the list of its earlier priority). So, with yield, p will have to wait until all other 40 priority processes finish their job or yield instead of just continuing as the active process. Levente > _,,,^..^,,,_ > best, Eliot > > |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
In reply to this post by Eliot Miranda-2
I may be completely off but let me ask you this:
How a process coming down from a higher priority should be treated: (A) as a 'regular' process trying to run at a given priority - i.e. send it to the back of the queue (B) as a sort of 'privileged' process deserving the front of the queue, as if previously 'preempted' by itself Now, as I can see only two usage examples - #valueUnpreemptively and #valueAt, the question is how are these two supposed to work ideally? At the moment they behave - or rather were supposed to I guess - like (A) and changing #yield implementation and using it in the #priority: method would preserve the (A) behavior. However, I wouldn't rule out behavior (B) because it would provide a nice choice based on #processPreemptionYields setting: - for `false` (default) the active process wouldn't yield and "stay" at the front of the queue and - for `true` the active process would yield and go to the end of the queue. For this scenario something like`[ ] forkAt: Processor highestPriority` in the #priority would provide such behavior. To illustrate such scenario I propose the following test: - currently it fails as the val assignment never happens... testPriority "test whether #priority: preempts active process to allow higher priority processes run; #priority behavior reflects the processPreemptionYields setting - for false (default) the active process gets to the front of the queue while for true it goes to the end" | val oldPriority | val := nil. oldPriority := Processor activePriority. Processor activeProcess priority: oldPriority + 2. [ val := false ] forkAt: oldPriority + 1. [ val := true ] forkAt: oldPriority. Processor activeProcess priority: oldPriority. self assert: val equals: Smalltalk vm processPreemptionYields Sorry for the lengthy message... Thanks -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
Re: The Inbox: Kernel-jar.1368.mcz
|
|
In reply to this post by Levente Uzonyi
Hi Levente, On Wed, Feb 10, 2021 at 11:54 AM Levente Uzonyi <[hidden email]> wrote: On Wed, 10 Feb 2021, Eliot Miranda wrote: But if that's the behaviour you want you can either - set the priority, do the processing, and then invoke yield - do the processing and then set the priority and then yield, right? We're not tying assigning to priority to the yield primitive, right? These are still distinct operations. I'm simply suggesting that the yield primitive take into account the receiver's priority, which it currently doesn't do.
_,,,^..^,,,_ best, Eliot |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
Hi, I sense a few different approaches here:
- fix #yield and leave #priority be as it is - fix #yield and use it to fix #priority - fix #yield and fix #priority independently of each other (my preference) best, Jaromir -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
Re: The Inbox: Kernel-jar.1368.mcz
|
|
In reply to this post by Eliot Miranda-2
Hi Eliot,
On Wed, 10 Feb 2021, Eliot Miranda wrote: > Hi Levente, > > On Wed, Feb 10, 2021 at 11:54 AM Levente Uzonyi <[hidden email]> wrote: > On Wed, 10 Feb 2021, Eliot Miranda wrote: > > > > > > > On Wed, Feb 10, 2021 at 11:27 AM Jaromir <[hidden email]> wrote: > > > the right solution is to create a new primitive if we want to support > > on-the-fly priority changes. > > > > The system already does on-the-fly priority changes - implementing > > #valueUnpreemptively and #valueAt: - but they're actually not working > > properly due to this #priority: issue - they're trying to use #yield after > > changing the priority down but that really doesn't do anything. > > > > > > So it seems to me that updating the yield primitive to "do the right thing" is a good way to go. Agreed? If so, I should be able to get to that very soon. > > I think it's not. Let's say the priority of process p is 30, I want to > increase it to 40 and then do something at that priority. > #yield will stop the process and put it at the end of the list of > processes with 40 priority (I think the current implementation doesn't > check for priority changes and puts the process on the list of its > earlier priority). > So, with yield, p will have to wait until all other 40 priority processes > finish their job or yield instead of just continuing as the active > process. > > > But if that's the behaviour you want you can either > - set the priority, do the processing, and then invoke yield > - do the processing and then set the priority and then yield, right? > > We're not tying assigning to priority to the yield primitive, right? These are still distinct operations. I'm simply suggesting that the yield primitive take into account the receiver's priority, which it currently doesn't do. from 42 to 40, and there's a process waiting to be executed at priority 41, then that process will not be activated but the process with priority 40 will keep on running. And, even yielding after the priority decrease doesn't help now. IMO, the ideal solution would be to 1. Make sure that #yield lets all existing higher priority processes run besides all the same priority processes. 2. When #priority: is sent to the active process to decrease its priority, then that process should a) should keep running without interruption if there's no higher priority processes awaiting execution than the new priority b) should be suspended if there are higher priority processes ready to run, and it should be resumed once all higher priority processes are suspended or terminated. Only fixing #yield does not help with the requirements in point 2. Levente > > > > Levente > > > _,,,^..^,,,_ > > best, Eliot > > > > > > > > -- > _,,,^..^,,,_ > best, Eliot > > |
|
|
On 11/02/21 5:10 am, Levente Uzonyi wrote:
> IMO, the ideal solution would be to > 1. Make sure that #yield lets all existing higher priority processes run > besides all the same priority processes all existing *equal* or higher priority ... yield just surrenders the time slot ahead of its expiry. The active process is moved to the back of its priority queue for #reschedule. > 2. When #priority: is sent to the active process to decrease its > priority, then that process should > a) should keep running without interruption if there's no higher > priority processes awaiting execution than the new priority > b) should be suspended if there are higher priority processes ready > to run, and it should be resumed once all higher priority processes are > suspended or terminated. Rescheduling logic should be separated from priority setting logic. #priority: should simply switch the process from its existing priority queue to the back of the new priority queue and then #reschedule. #yield reduces to setting a process to its current priority. It gives an opportunity for rescheduling. #reschedule logic should pick the next process to run. If this happens to be same as active process, no further change is required. Otherwise, the active process is suspended and the new process resumed. The third case when rescheduling is requested is by an async event like timer, interrupt, input etc. Here if the active process has exhausted its time slice, then it should be forced to #yield (fair share policy). Rescheduling a process is quite tricky. The working priority used in scheduling decisions could be different from that of a process's priority setting. Sometimes, a process (A) could be holding a mutex on which a higher priority process (H) is blocked. In this case, A's working priority is the highest priority of its waiters. HTH .. Subbu |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
In reply to this post by Levente Uzonyi
Hi Levente, On Wed, Feb 10, 2021 at 3:41 PM Levente Uzonyi <[hidden email]> wrote: Hi Eliot, Agreed. This means that yield should check to see if there is a runnable process at higher priority than the current process. It doesn't do this. It merely checks if there are processes at the same priority as the receiver. 2. When #priority: is sent to the active process to decrease its priority, Ah, yes, I see. That's tricky. "Fixing" yield to ensure that the highest priority process is runnable and using it in Process>>priority: would have the unwanted side-effect of yielding. So we may need another primitive. A primitive that answered the highest runnable priority process would work. Then we could write Process>>priority: anInteger priority := anInteger. ProcessorScheduler highestPriorityRunnableProcess priority > priority ifTrue: [self yield] That would be OK right? Levente _,,,^..^,,,_ best, Eliot |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
This post was updated on .
Hi,
> The current issue is that when a process's priority is decreased, e.g. > from 42 to 40, and there's a process waiting to be executed at priority > 41, then that process will not be activated but the process with priority > 40 will keep on running. > And, even yielding after the priority decrease doesn't help now. > > IMO, the ideal solution would be to > 1. Make sure that #yield lets all existing higher priority processes run > besides all the same priority processes. > 2. When #priority: is sent to the active process to decrease its priority, > then that process should > a) should keep running without interruption if there's no higher > priority processes awaiting execution than the new priority > b) should be suspended if there are higher priority processes > ready to run, and it should be resumed once all higher priority processes > are suspended or terminated. (b) is still unclear to me: if you decrease an active process's priority from 42 to 40 and there's a process waiting at 41, AND there's another process waiting at 40 then: the active process should suspend, the process at 41 should run and then what: resume your original process or run the other waiting process at 40 first? > Ah, yes, I see. That's tricky. "Fixing" yield to ensure that the highest > priority process is runnable and using it in Process>>priority: would have > the unwanted side-effect of yielding. So we may need another primitive. > A primitive that answered the highest runnable priority process would > work. Then we could write > > Process>>priority: anInteger > priority := anInteger. > ProcessorScheduler highestPriorityRunnableProcess priority > priority > ifTrue: > [self yield] > > That would be OK right? If the process should run before other processes waiting at its new (decreased) priority then #yield would still send the process to the end of the 40 queue, right? Btw, there's already a #nextReadyProcess method available but non-primitive so I understand it's not usable for the above purpose as such. The condition above should apply to the active process only, is that right? Process>>priority: anInteger priority := anInteger. self isActiveProcess and: [Processor nextReadyProcess priority > priority] ifTrue: ... best, Jaromir -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
Re: The Inbox: Kernel-jar.1368.mcz
|
|
In reply to this post by Eliot Miranda-2
Hi Eliot,
On Wed, 10 Feb 2021, Eliot Miranda wrote: > Hi Levente, > > On Wed, Feb 10, 2021 at 3:41 PM Levente Uzonyi <[hidden email]> wrote: > Hi Eliot, > > On Wed, 10 Feb 2021, Eliot Miranda wrote: > > > Hi Levente, > > > > On Wed, Feb 10, 2021 at 11:54 AM Levente Uzonyi <[hidden email]> wrote: > > On Wed, 10 Feb 2021, Eliot Miranda wrote: > > > > > > > > > > > On Wed, Feb 10, 2021 at 11:27 AM Jaromir <[hidden email]> wrote: > > > > the right solution is to create a new primitive if we want to support > > > on-the-fly priority changes. > > > > > > The system already does on-the-fly priority changes - implementing > > > #valueUnpreemptively and #valueAt: - but they're actually not working > > > properly due to this #priority: issue - they're trying to use #yield after > > > changing the priority down but that really doesn't do anything. > > > > > > > > > So it seems to me that updating the yield primitive to "do the right thing" is a good way to go. Agreed? If so, I should be able to get to that very soon. > > > > I think it's not. Let's say the priority of process p is 30, I want to > > increase it to 40 and then do something at that priority. > > #yield will stop the process and put it at the end of the list of > > processes with 40 priority (I think the current implementation doesn't > > check for priority changes and puts the process on the list of its > > earlier priority). > > So, with yield, p will have to wait until all other 40 priority processes > > finish their job or yield instead of just continuing as the active > > process. > > > > > > But if that's the behaviour you want you can either > > - set the priority, do the processing, and then invoke yield > > - do the processing and then set the priority and then yield, right? > > > > We're not tying assigning to priority to the yield primitive, right? These are still distinct operations. I'm simply suggesting that the yield primitive take into account the receiver's priority, which it currently doesn't do. > > The current issue is that when a process's priority is decreased, e.g. > from 42 to 40, and there's a process waiting to be executed at priority > 41, then that process will not be activated but the process with priority > 40 will keep on running. > And, even yielding after the priority decrease doesn't help now. > > IMO, the ideal solution would be to > 1. Make sure that #yield lets all existing higher priority processes run > besides all the same priority processes. > > > Agreed. This means that yield should check to see if there is a runnable process at higher priority than the current process. It doesn't do this. It merely checks if there are processes at the same priority as the receiver. process should ever be ready to run when #yield is sent. The problem only occurs when the active process's priority is decreased and there is a higher priority process awaiting execution. So, IMO, the right solution is to create a new primitive to set the process priority. When it is changing the active processes priority and the priority is decreased then look for higher priority processes awaiting execution. In other cases, just set the value of the instance variable. Levente > 2. When #priority: is sent to the active process to decrease its priority, > then that process should > a) should keep running without interruption if there's no higher > priority processes awaiting execution than the new priority > b) should be suspended if there are higher priority processes > ready to run, and it should be resumed once all higher priority processes > are suspended or terminated. > > Only fixing #yield does not help with the requirements in point 2. > > > Ah, yes, I see. That's tricky. "Fixing" yield to ensure that the highest priority process is runnable and using it in Process>>priority: would have the unwanted side-effect of yielding. So we may need another primitive. A primitive that answered the highest runnable priority process would work. Then we could > write > > Process>>priority: anInteger > priority := anInteger. > ProcessorScheduler highestPriorityRunnableProcess priority > priority ifTrue: > [self yield] > > That would be OK right? > > Levente > > > > > > > > > Levente > > > > > _,,,^..^,,,_ > > > best, Eliot > > > > > > > > > > > > > > -- > > _,,,^..^,,,_ > > best, Eliot > > > > > > > > -- > _,,,^..^,,,_ > best, Eliot > > |
Re: The Inbox: Kernel-jar.1368.mcz
|
|
In
http://forum.world.st/valueNoContextSwitch-no-different-from-value-td5127491.html I tested #valueNoContextSwitch and accidentally verified again how badly broken #valueUnpreemptively is due to the #priority bug. I also tried to apply the workaround I suggested in this post ([] forkAt: Processor highestPriority) and it's desperately slow so I can only agree a new primitive is probably the only good solution. I dont't think we necessarily need a whole #priority as a primitive though. A primitive where the scheduler gives an active process a kick to *preempt* when downgrading its priority should suffice: Process>>priority: anInteger self isActiveProcess ifTrue: [primitivePriority: anInteger] ... (so it's not the same as yielding; it's like preempting: take #valueUnpreemptively as an example - a process raises its priority and later goes back where it was - I guess the desired behavior is the process continues running after the downgrade (and no more higher prio processes present) even if there are processes waiting at the lower priority; they may have there even before the process raised its priority, so why send it to the back :) . I guess this Kernel-jar.1368.mcz can be removed from The Inbox. regards, ----- ^[^ Jaromir -- Sent from: http://forum.world.st/Squeak-Dev-f45488.html
^[^ Jaromir
|
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |