Hello,
Yesterday I needed to debug a process that is not the UI process. At one point this background process waits on a Promise (so it will wait on a Semaphore) that is supposed to be resolved in the UI process, so the background process will block. If this point is reached while stepping in the debugger, the UI process will block instead, since it evaluates the debugger stepping on behalf of the background process. What I did for now is to add an accessor on ProcessScheduler that answers the activeProcess without sending #effectiveProcess, and check in Promise>>wait whether this real active process is the UI process. If it is, halt the debugged process before the Semaphore wait primitive, and I proceed manually as soon as I am confident that the promise is already resolved or rejected, so the wait will return immediately. wait "Wait unconditionally for this promise to become fulfilled or rejected." | sema | (state = #pending and: [Processor realProcess == Project uiProcess]) ifTrue: [self halt]. sema := Semaphore new. ... But of course this is just a workaround. Is there a chance to extend the Squeak debugger to deal with critical sections and waits? In Eclipse for example, there is a list of threads; blocked/waiting threads are annotated with an icon, as well as the top frame that waits for something; you can debug other threads in the meantime. But in Java the debugger is in another VM than the debugged process, so the situation that the debugger steps in the UI process needs not be resolved there in the first place. Maybe if we know all the points/primitives where processes can be blocked, we could make the stepping detect these, run them in another process, make the debugger observe that process and show some info about the waiting in a special top context in the meantime. Once the other process is unblocked (or terminated if it does nothing else than the wait), the debugger would continue to step the background process. Could this work? Or is there already another facility that works and which I should use? Kind regards, Jakob |
Hi Jakob,
first, this seems like a really important issue that if addressed would improve the debugger a lot. So in my asking questions I’m not trying to knock dune your proposals, only trying to understand the problem accurately. > On Jul 23, 2020, at 5:50 AM, Jakob Reschke <[hidden email]> wrote: > > Hello, > > Yesterday I needed to debug a process that is not the UI process. At > one point this background process waits on a Promise (so it will wait > on a Semaphore) that is supposed to be resolved in the UI process, so > the background process will block. If this point is reached while > stepping in the debugger, the UI process will block instead, since it > evaluates the debugger stepping on behalf of the background process. Can you describe the control flow when this occurs normally, outside the debugger? I need to understand which processes execute waits. > What I did for now is to add an accessor on ProcessScheduler that > answers the activeProcess without sending #effectiveProcess, and check > in Promise>>wait whether this real active process is the UI process. > If it is, halt the debugged process before the Semaphore wait > primitive, and I proceed manually as soon as I am confident that the > promise is already resolved or rejected, so the wait will return > immediately. An alternative might be to have the debugger spawn a new UI process whenever it detects that the UI process is blocked. This may have to be done by a background helper process that the debugger spawns or allows to run whenever the debugger steps, because of course the ui process is executing the step. Without thinking things through, another alternative might be to have the debugger always use its own process. The underlying issue is that if the ui process blocks in reality then it should block in the debugger, but we’d like to keep the debugger alive and we’d like to keep the ui alive. So I suppose we could introduce a background process whose job it is to observe the ui process and if it detects the ui process is blocked for some time (1 second?) spawns a new ui process had reports (in the Transcript?, a pop-up in the new ui process?) that the ui process blocked. > > wait > "Wait unconditionally for this promise to become fulfilled or rejected." > | sema | > (state = #pending and: [Processor realProcess == Project > uiProcess]) ifTrue: [self halt]. > sema := Semaphore new. > ... > > But of course this is just a workaround. Is there a chance to extend > the Squeak debugger to deal with critical sections and waits? I don’t see how. The debugger can spot a critical section entry or a wait if one is doing “send”, because then the debugger actually executes the primitive in Context>>#tryPrimitive:..., (IIRC, I’m on my phone). But if the debugger steps then the critical section entry or wait will be executed by running code (the debugger runs step at full speed by using perform: and the code executed by the perform, while surrounded by an exception handler and an unwind-protect, is not virtualised in any way and so we have no easy way of intercepting critical section entry or waits). > In > Eclipse for example, there is a list of threads; blocked/waiting > threads are annotated with an icon, as well as the top frame that > waits for something; you can debug other threads in the meantime. But > in Java the debugger is in another VM than the debugged process, so > the situation that the debugger steps in the UI process needs not be > resolved there in the first place. > > Maybe if we know all the points/primitives where processes can be > blocked, we could make the stepping detect these, run them in another > process, make the debugger observe that process and show some info > about the waiting in a special top context in the meantime. Once the > other process is unblocked (or terminated if it does nothing else than > the wait), the debugger would continue to step the background process. > Could this work? > > Or is there already another facility that works and which I should use? As I say, I think a way that can work is detecting that the ui process has blocked and spawning a new ui process when that happens. But when in the debugger we want to avoid the debugger blocking in step/send and hence only executing any actions before step/send and not those after step/send (eg removing exception handler and unwind-protect from the stack). So architecturally, not having the debugger use the ui process and to use its own process seems like a good idea but a major change. > Kind regards, > Jakob Cheers, Eliot |
Hi Eliot,
Am Do., 23. Juli 2020 um 16:37 Uhr schrieb Eliot Miranda <[hidden email]>: > > > On Jul 23, 2020, at 5:50 AM, Jakob Reschke <[hidden email]> wrote: > > > > Hello, > > > > Yesterday I needed to debug a process that is not the UI process. At > > one point this background process waits on a Promise (so it will wait > > on a Semaphore) that is supposed to be resolved in the UI process, so > > the background process will block. If this point is reached while > > stepping in the debugger, the UI process will block instead, since it > > evaluates the debugger stepping on behalf of the background process. > > Can you describe the control flow when this occurs normally, outside the debugger? I need to understand which processes execute waits. > There is a "background" process that implements a certain workflow. There are one or more occasions during this workflow where user input is required. To move away from modal dialogs, nesting the world loop, and from writing all the business logic in a promise-style control flow, I want this background process to suspend until the user has provided the input (which happens in the UI process of course). So: 1. The background process triggers a request in the UI process via addDeferredUIMessage: or similar. Mind that this is not a modal invocation, so it will return immediately. 2. The background process waits for the input. In this particular case using Promise>>wait, which uses a Semaphore internally. 3. When the user is finished filling in the data, clicking the OK or Cancel buttons (which happens in the UI process) must somehow signal the background process to continue. In my current case, this happens by fulfilling the Promise on which the background process waits. This will signal the internal Promise Semaphore from the UI process. 4. The background process receives the signal on the Semaphore. It resumes to get the user's input from the Promise and continues the workflow. Instead of a Promise, other mechanisms could be used as well: - Use a Semaphore directly without Promise. - Let the background process read from a shared queue. - Let the background process suspend itself and be resumed from the UI callback explicitly. > > What I did for now is to add an accessor on ProcessScheduler that > > answers the activeProcess without sending #effectiveProcess, and check > > in Promise>>wait whether this real active process is the UI process. > > If it is, halt the debugged process before the Semaphore wait > > primitive, and I proceed manually as soon as I am confident that the > > promise is already resolved or rejected, so the wait will return > > immediately. > > An alternative might be to have the debugger spawn a new UI process whenever it detects that the UI process is blocked. This may have to be done by a background helper process that the debugger spawns or allows to run whenever the debugger steps, because of course the ui process is executing the step. > > Without thinking things through, another alternative might be to have the debugger always use its own process. The underlying issue is that if the ui process blocks in reality then it should block in the debugger, but we’d like to keep the debugger alive and we’d like to keep the ui alive. > > So I suppose we could introduce a background process whose job it is to observe the ui process and if it detects the ui process is blocked for some time (1 second?) spawns a new ui process had reports (in the Transcript?, a pop-up in the new ui process?) that the ui process blocked. > This sounds good at first and for my particular case. But in general this might break some things even outside of a debugging session: modal dialogs are allergic to switching the UI process (I guess everyone has seen Monticello's "This modal dialog was interrupted, please close it" message at some point). The other thing I can imagine is that after blocking a few seconds the no-longer-UI process might do something that needs to be done in the UI process, but won't use addDeferredUIMessage: because the code was designed to run in the UI process in the first place. If we can get around this, an always responsive UI would be great of course. > > > > wait > > "Wait unconditionally for this promise to become fulfilled or rejected." > > | sema | > > (state = #pending and: [Processor realProcess == Project > > uiProcess]) ifTrue: [self halt]. > > sema := Semaphore new. > > ... > > > > But of course this is just a workaround. Is there a chance to extend > > the Squeak debugger to deal with critical sections and waits? > > I don’t see how. The debugger can spot a critical section entry or a wait if one is doing “send”, because then the debugger actually executes the primitive in Context>>#tryPrimitive:..., (IIRC, I’m on my phone). But if the debugger steps then the critical section entry or wait will be executed by running code (the debugger runs step at full speed by using perform: and the code executed by the perform, while surrounded by an exception handler and an unwind-protect, is not virtualised in any way and so we have no easy way of intercepting critical section entry or waits). > We could add an AboutToWait notification around the code that calls the primitive... but it would slow down processes that don't need it. And if you add another blocking mechanism and forget to signal the notification, the debugger will block again. > > In > > Eclipse for example, there is a list of threads; blocked/waiting > > threads are annotated with an icon, as well as the top frame that > > waits for something; you can debug other threads in the meantime. But > > in Java the debugger is in another VM than the debugged process, so > > the situation that the debugger steps in the UI process needs not be > > resolved there in the first place. > > > > Maybe if we know all the points/primitives where processes can be > > blocked, we could make the stepping detect these, run them in another > > process, make the debugger observe that process and show some info > > about the waiting in a special top context in the meantime. Once the > > other process is unblocked (or terminated if it does nothing else than > > the wait), the debugger would continue to step the background process. > > Could this work? > > > > Or is there already another facility that works and which I should use? > > As I say, I think a way that can work is detecting that the ui process has blocked and spawning a new ui process when that happens. But when in the debugger we want to avoid the debugger blocking in step/send and hence only executing any actions before step/send and not those after step/send (eg removing exception handler and unwind-protect from the stack). So architecturally, not having the debugger use the ui process and to use its own process seems like a good idea but a major change. > Agreed. |
Hi Jakob, I have asked you several times to comment on the Promise
implementation in PromisesLocal, but you have not responded. This
is unfortunate.
Although the Promise defined in Kernel-Processes has #wait protocol, this is really an anathema to Promise-style control flow. The #whenResolved: protocol should be used to register a reactor and no waits should occur. One issue immediately apparent is that there is no event loop (In
the PriorityVat) that secures single-threaded behavior in the
promise. In PromisesLocal there is a LocalResolver who forwards
all pending #whenResolved: messages to the new reference, on the
event-loop Process (calls #redirectEventualMessage:). See ERef #promiseInVat: for the pair production of an asociation
with a PromiseERef and a LocalResolver. There is no equivalent to
this in Kernel-Processes. Promise-style control flow does not use mutexes nor semaphores. So I believe the fundamental model here is misused. You said: On 7/23/20 12:57 PM, Jakob Reschke
wrote:
To move away from modal dialogs, nesting the world loop, and from writing all the business logic in a promise-style control flow So it is your intention to "move away from promise-style control flow." then I would suggest not using a Promise. Kindly, |
Hi Jakob, I always want to assume best intentions, so I apologize for being
somewhat snarky in my last post. I am not offended nor am I mad at
you. Please don't be with me. On 7/23/20 2:00 PM, Robert Withers
wrote:
In consideration of my promises, I realize you may not understand
what I am proposing with the PromisesLocal package. It is a robust
amount of functional code. PromisesLocal attempts to be a working
implementation of the behaviors from ERights' ELib Concurrency
[1], which predates Mark Miller's joining Google Research for the
new JavaScript standard with his Promises/A+. I believe now that
the differences between the Promise in Kernel-Proccesses and
PromisesLocal is that PromisesLocal solves the concurrency issues,
with its concurrency model. Read about this here: [1] ELib's concurrency - http://www.erights.org/elib/concurrency/index.html So allow me to demonstrate the outcomes of using PromisesLocal. First image, taking a look at the Workspace, in the lower left
above the Transcript, we have a good send, an exception send, and
then another good send, demonstrating that the event loop does not
get blocked. The first and last explorers resolve to 420. The
middle explorer becomes a BrokenERef, due to the ZeroDivide. As
well there is a debugger open for this exception, opened on a
stack copy of the signaler context. The event loop continues to
run but we have another process opened in the debugger.
I am calling this in EventualMessageSend>>#value if an
exception occurs during the send. Second image is of the situation after proceeding from the
Debugger and then running the first line a second time, getting
another NearERef of 42. The event-loop is active. The third image demonstrates the PriorityVat running
#nextProcessMsg (the event loop!) as well as another process for
the ZeroDivide exception and its copied stack. Finally, allow me to add a halt in the Vat's machinery to halt
when an exception is handled in EventualMessageSend>>#value,
this will halt on the ZeroDivide, after resolving the promise to
broken, but before the stack copy of the signalerContext. This is
before the helperProcess is made, so we see in the Processes there
is the event loop open in the debugger (look for the process with
a priority of 30). As this broke before the last item in the
normalQ of the vat, the last explorer still shows a Promise. The
event loop is blocked. When the Debugger is resumed or abandoned,
the event loop DIES a quiet death, so we must clear it an restart
the #local Vat, to regain operability, thus the new first line in
the Workspace. I believe that adding a strategic call to
#ifCurtailed: would be helpful here, to restart the Vat. i am
unsure where to put this call. I hope the workspace examples give you more familiarity of use and these images show what is actually happening with PromisesLocal. Kindly, |
In reply to this post by Jakob Reschke
Hi all --
There is no explicit support for (coherent) multi-process debugging in Squeak's debugging and code simulation infrastructure (Debugger, Process, Context) yet. As a first step, we could clarify the meaning of stepping into or over a "semaphore wait" in any process. At the moment, the debugger (i.e. the current UI process) blocks. Well, even if the debugged process would block instead -- and not the current UI process -- how should the debugger react? Inform the user? Disable all stepping buttons? Add an extra watchdog to notify the debugger when the debugged process becomes responsive and hence "simulate-able" again? One challenging thing is that we cannot simulate the *entire* code execution in Squeak; and we don't have a clear agreement on how to deal with primitive calls during code simulation. Could we always use the fallback code? Hmmm.... Best, Marcel
|
Am Mo., 27. Juli 2020 um 11:19 Uhr schrieb Marcel Taeumel
<[hidden email]>: > > Could we always use the fallback code? Hmmm.... > Would it help in this particular situation? "Primitive failed" is "not fully functional" compared to a real Semaphore>>wait. And I doubt that it would be right to put in some debugging stub in the fallback implementation (e. g. a busy wait), though it might be okay. So the general question would be: what to do when essential primitives are encountered? And in particular the ones that will block... |
In reply to this post by Squeak - Dev mailing list
Forward reply to the list... seems like something has changed in the
Gmail interface and I have not yet adapted to it. Am Do., 23. Juli 2020 um 20:01 Uhr schrieb Robert Withers <[hidden email]>: > > Although the Promise defined in Kernel-Processes has #wait protocol, this is really an anathema to Promise-style control flow. The #whenResolved: protocol should be used to register a reactor and no waits should occur. > [...] > Promise-style control flow does not use mutexes nor semaphores. So I believe the fundamental model here is misused. > [...] > So it is your intention to "move away from promise-style control flow." then I would suggest not using a Promise. > That's right, but right now I have some dialogs implemented in a Promise style and want to implement a use case without promise style, so I have to literally adapt when invoking the dialog if I don't want to write all of it anew immediately. Our current Kernel Promise is a two-headed beast: for one it wants to be a Promises/A+ implementation, but originally it's purpose seems to be only to support #future sends. Promises/A+ is targeted at asynchrony in single-thread languages like JavaScript, while futures are for multi-thread languages like Smalltalk. So Kernel Promise now tries to serve both single-thread computing (UI process only) and multi-thread computing (multiple Smalltalk processes). |
In reply to this post by Jakob Reschke
Hi Jakob. Just think of two processes: consumer and producer -- shared queue with semaphore in between. How can we have two debuggers that enable us to step each process alternatingly. Your case with the UI process is just a special instance of an overall challenge we might want to address in Squeak's debugging infrastructure. I think that we added #effectiveProcess only for process-local variables? Or did we already tackle some semaphore/mutex issues with it? ... maybe the debugger should consider whether a process could be scheduled at all? I don't think that Squeak's processes encode this state at the moment. They are either scheduled or not. The latter could mean anything: waiting for a semaphore, manually suspended for debugging, ... Best, Marcel
|
In reply to this post by Jakob Reschke
Hi Jakob,
On 7/28/20 3:50 PM, Jakob Reschke wrote: > Forward reply to the list... seems like something has changed in the > Gmail interface and I have not yet adapted to it. > > Am Do., 23. Juli 2020 um 20:01 Uhr schrieb Robert Withers > <[hidden email]>: >> Although the Promise defined in Kernel-Processes has #wait protocol, this is really an anathema to Promise-style control flow. The #whenResolved: protocol should be used to register a reactor and no waits should occur. >> [...] >> Promise-style control flow does not use mutexes nor semaphores. So I believe the fundamental model here is misused. >> [...] >> So it is your intention to "move away from promise-style control flow." then I would suggest not using a Promise. >> > That's right, but right now I have some dialogs implemented in a > Promise style and want to implement a use case without promise style, > so I have to literally adapt when invoking the dialog if I don't want > to write all of it anew immediately. I am faced with a similar porting activity to bring SSL and SSH over onto the ThunkStack architecture. How do you dialogs function with a promise interface? Could you describe what you have done here, please? > Our current Kernel Promise is a two-headed beast: Interesting. > for one it wants to > be a Promises/A+ implementation, Even this is a concurrency model with #send:/#then: protocols. > but originally it's purpose seems to > be only to support #future sends. Which seems to have nothing at all to do with whether it is a Future. It's a Promise. This that you call future, is merely scheduling after a delay, right? > Promises/A+ is targeted at > asynchrony in single-thread languages like JavaScript, This is a definite concurrency protocol which allows for multi-agent interaction. I would suggest it is inherently a concurrency model, though it may not be explicit about single threaded vats. Smalltalk is also single-threaded on the processor. > while futures are for multi-thread languages like Smalltalk. I see the #then: protocol as the model for promises in multi-thread environments. Not the scheduling. > So Kernel Promise now > tries to serve both single-thread computing (UI process only) and > multi-thread computing (multiple Smalltalk processes). This multi-thread goal is the source of the #wait protocol, yes? I would like to suggest integrating PromisesLocal into the trunk image. I will send a separate email, describing my proposal. Kindly, rabbit |
Free forum by Nabble | Edit this page |