GC Crash/Bug due recursion?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

GC Crash/Bug due recursion?

Holger Freyther
Hi,

with the attached in progress grammar for a MGCP I end up with a crash when
marking OOPs (the code has a bug but probably should not crash the GC).

It probably crashes in the testReply, most likely because responseCode,
responseStringe and packageName have no proper return.


some backtrace.. it crashes because the stack overruns

#31 0x00f56c8a in _gst_mark_an_oop_internal (oop=<value optimized out>) at
oop.c:2262
#32 0x00f56c8a in _gst_mark_an_oop_internal (oop=<value optimized out>) at
oop.c:2262
#33 0x00f56c8a in _gst_mark_an_oop_internal (oop=<value optimized out>) at
oop.c:2262
#34 0x00f56c8a in _gst_mark_an_oop_internal (oop=<value optimized out>) at
oop.c:2262
#35 0x00f56c8a in _gst_mark_an_oop_internal (oop=<value optimized out>) at
oop.c:2262
#36 0x00f56c8a in _gst_mark_an_oop_internal (oop=<value optimized out>) at
oop.c:2262



_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

MGCPGrammar.st (6K) Download Attachment
MGCPGrammarTest.st (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Paolo Bonzini-2
On Fri, Jun 10, 2011 at 16:07, Holger Hans Peter Freyther
<[hidden email]> wrote:
> Hi,
>
> with the attached in progress grammar for a MGCP I end up with a crash when
> marking OOPs (the code has a bug but probably should not crash the GC).
>
> It probably crashes in the testReply, most likely because responseCode,
> responseStringe and packageName have no proper return.

Looks like an infinite loop in Smalltalk... Do you have enough stack
space to do a "call _gst_show_backtrace(stdout)" from gdb?

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Holger Freyther
On 06/10/2011 04:11 PM, Paolo Bonzini wrote:

> Looks like an infinite loop in Smalltalk... Do you have enough stack
> space to do a "call _gst_show_backtrace(stdout)" from gdb?

Hi,

yes, I just wonder if things should crash or if this can be prevented, it
seems to create an infinite loop in the GC which is weird... of course with
recursion in smalltalk the stack will be full too..


_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Paolo Bonzini-2
On Fri, Jun 10, 2011 at 16:15, Holger Hans Peter Freyther
<[hidden email]> wrote:

> On 06/10/2011 04:11 PM, Paolo Bonzini wrote:
>
>> Looks like an infinite loop in Smalltalk... Do you have enough stack
>> space to do a "call _gst_show_backtrace(stdout)" from gdb?
>
> Hi,
>
> yes, I just wonder if things should crash or if this can be prevented, it
> seems to create an infinite loop in the GC which is weird... of course with
> recursion in smalltalk the stack will be full too..

Yes, I think the problem is that the C stack of the marker is
exhausted before the whole Smalltalk stack is visited.

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

MrGwen
On 06/10/2011 04:27 PM, Paolo Bonzini wrote:

> On Fri, Jun 10, 2011 at 16:15, Holger Hans Peter Freyther
> <[hidden email]>  wrote:
>> On 06/10/2011 04:11 PM, Paolo Bonzini wrote:
>>
>>> Looks like an infinite loop in Smalltalk... Do you have enough stack
>>> space to do a "call _gst_show_backtrace(stdout)" from gdb?
>>
>> Hi,
>>
>> yes, I just wonder if things should crash or if this can be prevented, it
>> seems to create an infinite loop in the GC which is weird... of course with
>> recursion in smalltalk the stack will be full too..
>
> Yes, I think the problem is that the C stack of the marker is
> exhausted before the whole Smalltalk stack is visited.
>
> Paolo
>
> _______________________________________________
> help-smalltalk mailing list
> [hidden email]
> https://lists.gnu.org/mailman/listinfo/help-smalltalk
Hi,

I've made an iterative mark it works for long structure like :

Object subclass: Foo [
  | previous next|
  previous: anObject [
    previous := anObject
  ]
  next: anObject [
    next:= anObject
  ]
  next [
    ^ next
  ]
]

  | last |
last := Foo new.
1000000 timesRepeat: [ | f|
   f := Foo new.
   f previous: last.
   last next: f.
   last := last next. ]

This bench stresses well the it. mark patch and crashes the rec. mark.
But I guess there are still bugs with contexts (I should take a look),
there is a strange crash (DNU with #printNl lookup symbol is not found
but it's present...). But that's a good start :-)

It's possible instead of the iterative version to use a mark with a
pointer reversal but it's harder to debug, requires high time-cost:
visits each branch-node at least (n+1) times, each visit requires
additional memory fetches, and  each visit cycles 4 values +
reading/writing mark-flags.

Another solution is to use a stack guard (v8 does it) and checks
overflow Knuth propose something similar

Gwen

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

mark.patch (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Paolo Bonzini-2
On 06/21/2011 12:00 AM, Gwenael Casaccio wrote:

>
> This bench stresses well the it. mark patch and crashes the rec. mark.
> But I guess there are still bugs with contexts (I should take a look),
> there is a strange crash (DNU with #printNl lookup symbol is not found
> but it's present...). But that's a good start :-)
>
> It's possible instead of the iterative version to use a mark with a
> pointer reversal but it's harder to debug, requires high time-cost:
> visits each branch-node at least (n+1) times, each visit requires
> additional memory fetches, and  each visit cycles 4 values +
> reading/writing mark-flags.
>
> Another solution is to use a stack guard (v8 does it) and checks
> overflow Knuth propose something similar

Modulo the usual comments about irrelevant hunks :) it looks good,
thanks!  Can you however ensure the invariant that the queue is empty
upon entry to _gst_mark, by making it accept an OOP?  I believe this
simplifies the code and lets you add back the tail recursion
optimization you removed.

And it would be even better if you could keep the distinction between
marking one OOP and marking a range,  because it removes a lot of
traffic to/from the mark queue (a bit cheaper than the stack because of
no stack frame overhead, but still expensive cache-wise).  Which in turn
means, keep the API as is! :)

Anyway, good start!

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

MrGwen
On 06/27/2011 02:56 PM, Paolo Bonzini wrote:

> On 06/21/2011 12:00 AM, Gwenael Casaccio wrote:
>>
>> This bench stresses well the it. mark patch and crashes the rec. mark.
>> But I guess there are still bugs with contexts (I should take a look),
>> there is a strange crash (DNU with #printNl lookup symbol is not found
>> but it's present...). But that's a good start :-)
>>
>> It's possible instead of the iterative version to use a mark with a
>> pointer reversal but it's harder to debug, requires high time-cost:
>> visits each branch-node at least (n+1) times, each visit requires
>> additional memory fetches, and each visit cycles 4 values +
>> reading/writing mark-flags.
>>
>> Another solution is to use a stack guard (v8 does it) and checks
>> overflow Knuth propose something similar
>
> Modulo the usual comments about irrelevant hunks :) it looks good,
> thanks! Can you however ensure the invariant that the queue is empty
> upon entry to _gst_mark, by making it accept an OOP? I believe this
> simplifies the code and lets you add back the tail recursion
> optimization you removed.
>
> And it would be even better if you could keep the distinction between
> marking one OOP and marking a range, because it removes a lot of traffic
> to/from the mark queue (a bit cheaper than the stack because of no stack
> frame overhead, but still expensive cache-wise). Which in turn means,
> keep the API as is! :)
>
> Anyway, good start!
>
> Paolo
Is it better Paolo ?

Cheers,
Gwen

Btw
Thanks for all the time you spend for the review :)

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

mark.patch (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Paolo Bonzini-2
On 07/04/2011 12:48 PM, Gwenael Casaccio wrote:
> Is it better Paolo ?

What's the difference from v1?  I see nothing of what I asked for:

>> Modulo the usual comments about irrelevant hunks :) it looks good,
>> thanks! Can you however ensure the invariant that the queue is empty
>> upon entry to _gst_mark, by making it accept an OOP? I believe this
>> simplifies the code and lets you add back the tail recursion
>> optimization you removed.
>>
>> And it would be even better if you could keep the distinction between
>> marking one OOP and marking a range, because it removes a lot of traffic
>> to/from the mark queue (a bit cheaper than the stack because of no stack
>> frame overhead, but still expensive cache-wise). Which in turn means,
>> keep the API as is! :)

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

MrGwen
On 07/04/2011 01:07 PM, Paolo Bonzini wrote:

> On 07/04/2011 12:48 PM, Gwenael Casaccio wrote:
>> Is it better Paolo ?
>
> What's the difference from v1?  I see nothing of what I asked for:
>
>>> Modulo the usual comments about irrelevant hunks :) it looks good,
>>> thanks! Can you however ensure the invariant that the queue is empty
>>> upon entry to _gst_mark, by making it accept an OOP? I believe this
>>> simplifies the code and lets you add back the tail recursion
>>> optimization you removed.
>>>
>>> And it would be even better if you could keep the distinction between
>>> marking one OOP and marking a range, because it removes a lot of traffic
>>> to/from the mark queue (a bit cheaper than the stack because of no stack
>>> frame overhead, but still expensive cache-wise). Which in turn means,
>>> keep the API as is! :)
>
> Paolo
Sorry it was not the good patch.

Gwen

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

mark.patch (3K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Paolo Bonzini-2
What I asked:

>>> by making it accept an OOP

What you have:

> --- a/libgst/oop.c
> +++ b/libgst/oop.c
> @@ -2242,68 +2242,89 @@ _gst_add_an_oop_to_mark_queue (OOP oop)
>  void
>  _gst_mark (void)

What I asked:

>> What's the difference from v1?

What you have:

> Sorry it was not the good patch.

Please read messages _before_ replying to them.

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

MrGwen
On 07/04/2011 01:40 PM, Paolo Bonzini wrote:

> What I asked:
>
>>>> by making it accept an OOP
>
> What you have:
>
>> --- a/libgst/oop.c
>> +++ b/libgst/oop.c
>> @@ -2242,68 +2242,89 @@ _gst_add_an_oop_to_mark_queue (OOP oop)
>> void
>> _gst_mark (void)
>
> What I asked:
>
>>> What's the difference from v1?
>
> What you have:
>
>> Sorry it was not the good patch.
>
> Please read messages _before_ replying to them.
>
> Paolo
Hello,

I've made some changes:

1) I keep _gst_mark_an_oop_internal and removed
_gst_add_an_oop_to_mark_queue for adding an OOP in the Queue. So like
the previous one, it has an OOP has argument and visit it.

2) in _gst_mark_an_oop_internal when visiting one OOP I keep the same
behavior with TAIL_MARK_OOP, and TAIL_MARK_OOPRANGE add OOPs in the Queue.

3) _gst_mark_oop_range keeps the same behavior as the old one

The patch is tested and produced with master (all tests are green)

Gwen

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

mark.patch (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Paolo Bonzini-2
On 07/04/2011 09:42 PM, Gwenael Casaccio wrote:

>
> I've made some changes:
>
> 1) I keep _gst_mark_an_oop_internal and removed
> _gst_add_an_oop_to_mark_queue for adding an OOP in the Queue. So like
> the previous one, it has an OOP has argument and visit it.
>
> 2) in _gst_mark_an_oop_internal when visiting one OOP I keep the same
> behavior with TAIL_MARK_OOP, and TAIL_MARK_OOPRANGE add OOPs in the Queue.
>
> 3) _gst_mark_oop_range keeps the same behavior as the old one
>
> The patch is tested and produced with master (all tests are green)

Thanks, these are the minimum changes required.  I'll tweak it a bit to
remove the final push/pop and commit it.

Paolo

_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk
Reply | Threaded
Open this post in threaded view
|

Re: GC Crash/Bug due recursion?

Paolo Bonzini-2
On 07/05/2011 08:32 AM, Paolo Bonzini wrote:

> On 07/04/2011 09:42 PM, Gwenael Casaccio wrote:
>>
>> I've made some changes:
>>
>> 1) I keep _gst_mark_an_oop_internal and removed
>> _gst_add_an_oop_to_mark_queue for adding an OOP in the Queue. So like
>> the previous one, it has an OOP has argument and visit it.
>>
>> 2) in _gst_mark_an_oop_internal when visiting one OOP I keep the same
>> behavior with TAIL_MARK_OOP, and TAIL_MARK_OOPRANGE add OOPs in the
>> Queue.
>>
>> 3) _gst_mark_oop_range keeps the same behavior as the old one
>>
>> The patch is tested and produced with master (all tests are green)
>
> Thanks, these are the minimum changes required. I'll tweak it a bit to
> remove the final push/pop and commit it.
Here is what I committed.

Paolo


_______________________________________________
help-smalltalk mailing list
[hidden email]
https://lists.gnu.org/mailman/listinfo/help-smalltalk

0001-replace-recursion-with-mark-stack.patch (5K) Download Attachment