About the infinite debugger

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

About the infinite debugger

Guillermo Polito
Hi all,

during today's sprint we have been working with lots of people on the infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We have checked the emails sent in the latest month. Then, together with Quentin, Pablo, Pavel, Yoan we have been discussing and testing hypothesis all day. We have been also comparing the debuggers code between pharo 3/4 (where the bug was is present) and pharo 7, but this was not necessarily straight forward as the code is not the same and there is no easy diff...

AAAAND, we have found that the problem may come from a wrong pragma marker. Just removing that pragma gives us back the same behaviour as in  Pharo 3/4. :D


I know that the exception handling/debugging has been modified several times in the latest years (some refactorings, hiding contexts...), we unfortunately don't have tests for it, so I'd like some more pair of eyes on it. Ben, Martin could you take a look?

Thanks all for the fish,
Guille
Reply | Threaded
Open this post in threaded view
|

Re: [rmod] About the infinite debugger

Clément Bera-4
Hi,

Normally the pull request should break the exception tests, especially the ones with nested handler, since they are the reason 199 marked signalling contexts were introduced.

Looking quickly this test in latest Pharo 7 for example:
ExceptionTests>>testHandlerContext
fails with the pull request.

Since the pull request changes a green test into a red test, I don't understand why you say "we unfortunately don't have tests for it". I don't understand either why you say this pragma is "wrong", so I can't help with that either.

Best,

On Fri, Jun 29, 2018 at 4:48 PM, Guillermo Polito <[hidden email]> wrote:
Hi all,

during today's sprint we have been working with lots of people on the infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We have checked the emails sent in the latest month. Then, together with Quentin, Pablo, Pavel, Yoan we have been discussing and testing hypothesis all day. We have been also comparing the debuggers code between pharo 3/4 (where the bug was is present) and pharo 7, but this was not necessarily straight forward as the code is not the same and there is no easy diff...

AAAAND, we have found that the problem may come from a wrong pragma marker. Just removing that pragma gives us back the same behaviour as in  Pharo 3/4. :D


I know that the exception handling/debugging has been modified several times in the latest years (some refactorings, hiding contexts...), we unfortunately don't have tests for it, so I'd like some more pair of eyes on it. Ben, Martin could you take a look?

Thanks all for the fish,
Guille



--
Reply | Threaded
Open this post in threaded view
|

Re: About the infinite debugger

Ben Coman
In reply to this post by Guillermo Polito


On 29 June 2018 at 22:48, Guillermo Polito <[hidden email]> wrote:
Hi all,

during today's sprint we have been working with lots of people on the infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We have checked the emails sent in the latest month. Then, together with Quentin, Pablo, Pavel, Yoan we have been discussing and testing hypothesis all day. We have been also comparing the debuggers code between pharo 3/4 (where the bug was is present) and pharo 7, but this was not necessarily straight forward as the code is not the same and there is no easy diff...

AAAAND, we have found that the problem may come from a wrong pragma marker. Just removing that pragma gives us back the same behaviour as in  Pharo 3/4. :D


I know that the exception handling/debugging has been modified several times in the latest years (some refactorings, hiding contexts...), we unfortunately don't have tests for it, so I'd like some more pair of eyes on it. Ben, Martin could you take a look?

Thanks all for the fish,
Guille

I'm just about to fly out for a family vacation, so I'm not sure when I'll get a chance to have a good look.
But I read the issue on Fogbugz and it echos some things I've been thinking recently,
though I hadn't come close to that particular fix.

I think my following comments are probably unrelated, but just sharing some thoughts that crossed my mind when I was looking into my own similar trouble.
This is an area I'm not clear on and would like to understand better...

When the UI thread (UI1) needs to be debugged (e.g. code run from Playground, or a test)
and is suspended and a new UI thread (UI2) is started, which thread does the debugger run in?
I guess it must run in UI2 but sometimes I got the vague feeling some parts of it were caught in UI1 and affected by UI1 being suspended.
But perhaps I got the wrong impression - its a complex interplay to follow.

The other vague notion I had was when thread UI1 is resumed, it seems to replace thread UI2 where the World keeps its, 
and wondered if <StepOver> on the suspended-debug-UI1 thread somehow made it "resume" and so replace UI2 as the main thread, 
but then UI1 gets suspended again and the display freezes.

Thanks for everyones efforts on this.
cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: [rmod] About the infinite debugger

Guillermo Polito
In reply to this post by Clément Bera-4


On Fri, Jun 29, 2018 at 5:54 PM Clément Bera <[hidden email]> wrote:
Hi,

Normally the pull request should break the exception tests, especially the ones with nested handler, since they are the reason 199 marked signalling contexts were introduced

Yes, but it's ok that on:do: is marked but why evaluateSignal:? Do you know?
 
.

Looking quickly this test in latest Pharo 7 for example:
ExceptionTests>>testHandlerContext
fails with the pull request.

Since the pull request changes a green test into a red test, I don't understand why you say "we unfortunately don't have tests for it".

Because I haven't see those. I was looking for tests about #findNextHandlerContext, #findNextHandlerOrSignalingContext and I found nothing.
But if we have some tests much better!
 
I don't understand either why you say this pragma is "wrong", so I can't help with that either.

Well, I don't know if I have to repeat myself... This is apparently provoking the infinite debugger issue. This pragma was not there in Pharo3, and its not clear why a method that is not an exception handler is marked as an exception handler. Is that clear enough?

For more info you can read the issue.
 

Best,

On Fri, Jun 29, 2018 at 4:48 PM, Guillermo Polito <[hidden email]> wrote:
Hi all,

during today's sprint we have been working with lots of people on the infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We have checked the emails sent in the latest month. Then, together with Quentin, Pablo, Pavel, Yoan we have been discussing and testing hypothesis all day. We have been also comparing the debuggers code between pharo 3/4 (where the bug was is present) and pharo 7, but this was not necessarily straight forward as the code is not the same and there is no easy diff...

AAAAND, we have found that the problem may come from a wrong pragma marker. Just removing that pragma gives us back the same behaviour as in  Pharo 3/4. :D


I know that the exception handling/debugging has been modified several times in the latest years (some refactorings, hiding contexts...), we unfortunately don't have tests for it, so I'd like some more pair of eyes on it. Ben, Martin could you take a look?

Thanks all for the fish,
Guille



--


--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: About the infinite debugger

Guillermo Polito
In reply to this post by Ben Coman


On Fri, Jun 29, 2018 at 5:58 PM Ben Coman <[hidden email]> wrote:


On 29 June 2018 at 22:48, Guillermo Polito <[hidden email]> wrote:
Hi all,

during today's sprint we have been working with lots of people on the infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We have checked the emails sent in the latest month. Then, together with Quentin, Pablo, Pavel, Yoan we have been discussing and testing hypothesis all day. We have been also comparing the debuggers code between pharo 3/4 (where the bug was is present) and pharo 7, but this was not necessarily straight forward as the code is not the same and there is no easy diff...

AAAAND, we have found that the problem may come from a wrong pragma marker. Just removing that pragma gives us back the same behaviour as in  Pharo 3/4. :D


I know that the exception handling/debugging has been modified several times in the latest years (some refactorings, hiding contexts...), we unfortunately don't have tests for it, so I'd like some more pair of eyes on it. Ben, Martin could you take a look?

Thanks all for the fish,
Guille

I'm just about to fly out for a family vacation, so I'm not sure when I'll get a chance to have a good look.
But I read the issue on Fogbugz and it echos some things I've been thinking recently,
though I hadn't come close to that particular fix.

I think my following comments are probably unrelated, but just sharing some thoughts that crossed my mind when I was looking into my own similar trouble.
This is an area I'm not clear on and would like to understand better...

When the UI thread (UI1) needs to be debugged (e.g. code run from Playground, or a test)
and is suspended and a new UI thread (UI2) is started, which thread does the debugger run in?

Yes
 
I guess it must run in UI2 but sometimes I got the vague feeling some parts of it were caught in UI1 and affected by UI1 being suspended. 
But perhaps I got the wrong impression - its a complex interplay to follow.

Nope.. Actually, UI2 will simulate code of UI1 as it was running in UI1. There is some machinery (look at #effectiveProcess) so that 

Processor activeProcess

yields UI1 when stepping over UI2 code.
 

The other vague notion I had was when thread UI1 is resumed, it seems to replace thread UI2 where the World keeps its, 
and wondered if <StepOver> on the suspended-debug-UI1 thread somehow made it "resume" and so replace UI2 as the main thread, 
but then UI1 gets suspended again and the display freezes.

Thanks for everyones efforts on this.
cheers -ben


--

   

Guille Polito

Research Engineer

Centre de Recherche en Informatique, Signal et Automatique de Lille

CRIStAL - UMR 9189

French National Center for Scientific Research - http://www.cnrs.fr


Web: http://guillep.github.io

Phone: +33 06 52 70 66 13

Reply | Threaded
Open this post in threaded view
|

Re: About the infinite debugger

Esteban A. Maringolo
In reply to this post by Guillermo Polito
Great news!

I hope this fixes it once and for all.

For some reason this bug bit me several times in several Pharo releases.

Thank you all for your dedication.

On 29/06/2018 11:48, Guillermo Polito wrote:

> Hi all,
>
> during today's sprint we have been working with lots of people on the
> infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We
> have checked the emails sent in the latest month. Then, together with
> Quentin, Pablo, Pavel, Yoan we have been discussing and testing
> hypothesis all day. We have been also comparing the debuggers code
> between pharo 3/4 (where the bug was is present) and pharo 7, but this
> was not necessarily straight forward as the code is not the same and
> there is no easy diff...
>
> AAAAND, we have found that the problem may come from a wrong pragma
> marker. Just removing that pragma gives us back the same behaviour as
> in  Pharo 3/4. :D
>
> https://github.com/pharo-project/pharo/pull/1621
>
> I know that the exception handling/debugging has been modified several
> times in the latest years (some refactorings, hiding contexts...), we
> unfortunately don't have tests for it, so I'd like some more pair of
> eyes on it. Ben, Martin could you take a look?
>
> Thanks all for the fish,
> Guille

--
Esteban A. Maringolo

Reply | Threaded
Open this post in threaded view
|

Re: About the infinite debugger

Eliot Miranda-2
In reply to this post by Guillermo Polito
Hi Guille,

On Jun 29, 2018, at 7:48 AM, Guillermo Polito <[hidden email]> wrote:

Hi all,

during today's sprint we have been working with lots of people on the infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We have checked the emails sent in the latest month. Then, together with Quentin, Pablo, Pavel, Yoan we have been discussing and testing hypothesis all day. We have been also comparing the debuggers code between pharo 3/4 (where the bug was is present) and pharo 7, but this was not necessarily straight forward as the code is not the same and there is no easy diff...

This is frustrating.  I can’t see the issue cuz I can’t login to fogbugz.  Having to login to read an issue is a major flaw.  I can see it makes sense for submitting, but fir merely browsing it should be unacceptable.  That said...

The pragma <primitive: 199> actually sets the primitive number in the method, so it it not merely a pragma; it alters bits in the method that the VM uses to search for handler contexts.  So why one would do that for evaluateSignal: makes no sense to me. The primitive should be set only in on:do: or something very similar (for example one could imagine adding on:or:do: instead of using , to construct an ExceptionSet).  So I think removing it from evaluateSignal: is definitely the right thing to do.

As far as tests for findNextHandlerFrom:, this is tested implicitly by any nested exception test so I expect you have several tests affected.  Clément points to a test that fails when not including <primitive: 199> in evaluateSignal: so more investigation is necessary.  Difficult to do while bugs are hidden in fogbugz.  When are they going to migrate the github where they belong?


AAAAND, we have found that the problem may come from a wrong pragma marker. Just removing that pragma gives us back the same behaviour as in  Pharo 3/4. :D


I know that the exception handling/debugging has been modified several times in the latest years (some refactorings, hiding contexts...), we unfortunately don't have tests for it, so I'd like some more pair of eyes on it. Ben, Martin could you take a look?

Thanks all for the fish,
Guille
Reply | Threaded
Open this post in threaded view
|

Re: About the infinite debugger

Marcus Denker-4


On 30 Jun 2018, at 02:56, Eliot Miranda <[hidden email]> wrote:

Hi Guille,

On Jun 29, 2018, at 7:48 AM, Guillermo Polito <[hidden email]> wrote:

Hi all,

during today's sprint we have been working with lots of people on the infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/). We have checked the emails sent in the latest month. Then, together with Quentin, Pablo, Pavel, Yoan we have been discussing and testing hypothesis all day. We have been also comparing the debuggers code between pharo 3/4 (where the bug was is present) and pharo 7, but this was not necessarily straight forward as the code is not the same and there is no easy diff...

This is frustrating.  I can’t see the issue cuz I can’t login to fogbugz.  Having to login to read an issue is a major flaw.  I can see it makes sense for submitting, but fir merely browsing it should be unacceptable.  That said...

The pragma <primitive: 199> actually sets the primitive number in the method, so it it not merely a pragma; it alters bits in the method that the VM uses to search for handler contexts.  So why one would do that for evaluateSignal: makes no sense to me. The primitive should be set only in on:do: or something very similar (for example one could imagine adding on:or:do: instead of using , to construct an ExceptionSet).  So I think removing it from evaluateSignal: is definitely the right thing to do.

As far as tests for findNextHandlerFrom:, this is tested implicitly by any nested exception test so I expect you have several tests affected.  Clément points to a test that fails when not including <primitive: 199> in evaluateSignal: so more investigation is necessary.  Difficult to do while bugs are hidden in fogbugz.  When are they going to migrate the github where they belong?

Yes, the idea was to move the issue tracker to GitHub… but after the release is done (as it requires changing again some things).
The closed-ness of fugbugz is really annoying.

Marcus

Reply | Threaded
Open this post in threaded view
|

Re: About the infinite debugger

Martin McClure-2
In reply to this post by Guillermo Polito
On 06/29/2018 07:48 AM, Guillermo Polito wrote:

> Hi all,
>
> during today's sprint we have been working with lots of people on the
> infinite debugger problem (https://pharo.fogbugz.com/f/cases/22085/).
> We have checked the emails sent in the latest month. Then, together
> with Quentin, Pablo, Pavel, Yoan we have been discussing and testing
> hypothesis all day. We have been also comparing the debuggers code
> between pharo 3/4 (where the bug was is present) and pharo 7, but this
> was not necessarily straight forward as the code is not the same and
> there is no easy diff...
>
> AAAAND, we have found that the problem may come from a wrong pragma
> marker. Just removing that pragma gives us back the same behaviour as
> in  Pharo 3/4. :D
>
> https://github.com/pharo-project/pharo/pull/1621
>
> I know that the exception handling/debugging has been modified several
> times in the latest years (some refactorings, hiding contexts...), we
> unfortunately don't have tests for it, so I'd like some more pair of
> eyes on it. Ben, Martin could you take a look?
>

Hi Guille,
I'm just back from vacation last week, and about to go on vacation for
another week, but I'll see what I can see.
About the primitive pragmas for context-marking, I think some of those
were changed for the exception handling fix that Andres and I did a few
years back, so *could* be involved in this. I'd hate to see regression
in the exception handling in an attempt to fix this bug.

Regards,
-Martin

Reply | Threaded
Open this post in threaded view
|

Re: About the infinite debugger

Martin McClure-2
On 07/03/2018 05:02 PM, Martin McClure wrote:

> On 06/29/2018 07:48 AM, Guillermo Polito wrote:
>> I know that the exception handling/debugging has been modified several
>> times in the latest years (some refactorings, hiding contexts...), we
>> unfortunately don't have tests for it, so I'd like some more pair of
>> eyes on it. Ben, Martin could you take a look?
>>
> Hi Guille,
> I'm just back from vacation last week, and about to go on vacation for
> another week, but I'll see what I can see.
> About the primitive pragmas for context-marking, I think some of those
> were changed for the exception handling fix that Andres and I did a few
> years back, so *could* be involved in this. I'd hate to see regression
> in the exception handling in an attempt to fix this bug.
>

After a look at at the pull request, I'm quite sure that removing the
prim 199 marker is the wrong thing to do. Thanks, Pablo, for restoring
it! The start of execution of exception handlers must be marked in order
for #findNextHandlerContext to work correctly. If these contexts are not
marked, exceptions signaled from inside an exception handler can find
the wrong handler. See the code and comment in #findNextHandlerContext.

I'm afraid that I cannot immediately help with the debugger problem,
since I don't know the debugger nearly as well as I do the exception
handling code, and I'm going on vacation for a week in 20 minutes. :-)
Perhaps when I get back I can take a look at it if it is still a problem
by then.

Regards,
-Martin